Streaming

BindAI v0.1 provides a streaming endpoint for agent execution through the REST API. Streaming allows a client to receive agent output progressively instead of waiting for the complete response. The endpoint is:

POST /api/v1/agents/{agent_name}/stream

The endpoint requires API-key authentication.

Streaming Architecture

A streaming request follows this general flow:

Client
   |
   | POST /api/v1/agents/{agent_name}/stream
   v
BindAI API
   |
   v
Configured Agent
   |
   v
Agent Execution
   |
   v
Streaming HTTP Response
   |
   v
Client

The API uses FastAPI’s StreamingResponse to return the streamed output. The endpoint provides a lightweight HTTP streaming boundary around BindAI agent execution.

Endpoint

The streaming endpoint is:

POST /api/v1/agents/{agent_name}/stream

The {agent_name} parameter identifies the configured agent that should execute the request. For example:

POST /api/v1/agents/researcher/stream

The exact agent name depends on the agents configured in the BindAI application.

Authentication

The streaming endpoint is protected by the BindAI API authentication layer. Clients must provide:

Authorization: Bearer <API key>

For example:

curl \
  -H "Authorization: Bearer your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"message":"Explain retrieval augmented generation."}' \
  http://localhost:8000/api/v1/agents/researcher/stream

The configured API key is provided through:

BINDAI_API_KEY=your-secret-key

See the Authentication documentation for the complete authentication model.

Request Body

The streaming endpoint accepts an agent run request. The request body contains:

{
  "message": "Explain retrieval augmented generation."
}

The message field is required and must contain a non-empty string. The request model is the same request model used by the normal agent execution endpoint.

Streaming Response

The endpoint returns a streaming HTTP response. The response allows clients to consume agent output incrementally. Conceptually:

Agent starts
    |
    v
Output
    |
    v
Client receives streamed data
    |
    v
More output
    |
    v
Client receives more streamed data
    |
    v
Agent completes
    |
    v
Response ends

The exact amount of data delivered in each chunk depends on the underlying execution. Clients should therefore not assume a fixed number of chunks or a particular chunk size.

Streaming with cURL

A simple command-line example is:

curl \
  -N \
  -H "Authorization: Bearer your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"message":"Write a short explanation of AI agents."}' \
  http://localhost:8000/api/v1/agents/researcher/stream

The -N option tells cURL not to buffer the streamed response unnecessarily.

Streaming with Python

A Python HTTP client can consume the response incrementally. For example, using requests:

import requests

response = requests.post(
    "http://localhost:8000/api/v1/agents/researcher/stream",
    headers={
        "Authorization": "Bearer your-secret-key",
        "Content-Type": "application/json",
    },
    json={
        "message": "Explain AI agents in simple terms.",
    },
    stream=True,
)

response.raise_for_status()

for chunk in response.iter_content(
    chunk_size=None,
    decode_unicode=True,
):
    if chunk:
        print(chunk, end="", flush=True)

The client processes each received chunk as it arrives.

Streaming with JavaScript

A browser or server-side JavaScript application can consume the HTTP response using the Fetch API. For example:

const response = await fetch(
  "http://localhost:8000/api/v1/agents/researcher/stream",
  {
    method: "POST",
    headers: {
      "Authorization": "Bearer your-secret-key",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      message: "Explain AI agents in simple terms.",
    }),
  },
);

if (!response.ok) {
  throw new Error(`Request failed: ${response.status}`);
}

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { value, done } = await reader.read();

  if (done) {
    break;
  }

  const text = decoder.decode(value, { stream: true });

  console.log(text);
}

The client can append each received chunk to the application’s output interface.

Streaming vs Normal Execution

BindAI provides both normal and streaming agent execution. Normal execution:

POST /api/v1/agents/{agent_name}/run

Streaming execution:

POST /api/v1/agents/{agent_name}/stream

The conceptual difference is:

Normal

Client
  |
  v
Agent
  |
  v
Complete Result
  |
  v
Client

and:

Streaming

Client
  |
  v
Agent
  |
  +---- Streamed output
  +---- Streamed output
  +---- Streamed output
  +---- Streamed output
  |
  v
Client

Normal execution is convenient when the client needs the complete result before continuing. Streaming is useful when the client wants to display generated output incrementally.

When to Use Streaming

Streaming is particularly useful for interactive applications. Examples include:

Chat interfaces
Agent consoles
Interactive assistants
Long model responses
Generated explanations
Content generation
Developer tools

A user interface can display output while the agent is still producing it rather than waiting for the complete response.

Streaming and Model Providers

The REST API provides the HTTP streaming boundary around agent execution. The agent and its configured model provider determine how the underlying execution produces output. Conceptually:

REST API
   |
   v
Agent
   |
   v
Model Provider
   |
   v
Model

Different providers may produce output at different rates. Clients should therefore treat the response as an incremental stream rather than relying on provider-specific timing.

Chunk Handling

Clients should treat received data as part of a larger response. A chunk may not represent:

A complete sentence
A complete word
A complete semantic unit
A complete application-level event

For example:

Chunk 1: "Retrieval"
Chunk 2: " augmented"
Chunk 3: " generation"
Chunk 4: " combines"

A client should combine chunks when displaying the complete generated response. Do not build application logic that assumes each chunk is independently meaningful.

Buffering

Streaming can be affected by buffering at different layers. Potential buffering points include:

Model Provider
      |
      v
BindAI Application
      |
      v
Uvicorn
      |
      v
Reverse Proxy
      |
      v
Client

A reverse proxy or hosting platform may buffer responses even when the application itself is streaming. When deploying streaming applications, verify that the surrounding infrastructure supports streaming responses.

Reverse Proxies

If a reverse proxy is placed in front of BindAI:

Client
   |
   v
Reverse Proxy
   |
   v
BindAI API

the proxy must be configured appropriately for streaming traffic. Otherwise, the proxy may collect response data before forwarding it to the client. The exact configuration depends on the reverse proxy and hosting platform. BindAI does not control infrastructure-level response buffering.

Connection Lifetime

A streaming request remains active while the HTTP response is being produced. Conceptually:

Request
  |
  v
Agent starts
  |
  |------ stream active ------|
  |                           |
  v                           v
First output              Final output
                              |
                              v
                         Response ends

Clients should keep the HTTP connection open until the stream completes or an error occurs. Infrastructure should also use appropriate request and connection timeouts for streaming workloads.

Client Disconnects

A client may disconnect before the response has completed. For example:

Client
   |
   | Request
   v
BindAI API
   |
   v
Agent
   |
   +---- Output
   |
   X Client disconnects

Applications should not assume that the client will always remain connected for the entire execution. The behavior of an agent execution after a client disconnect depends on the underlying application and execution lifecycle. Streaming should therefore not be treated as a durable background execution mechanism. For work that must continue independently of the client connection, use the background run API instead.

Streaming vs Background Runs

Streaming and background execution solve different problems. Streaming:

Client
   |
   v
Agent
   |
   v
Live HTTP output

Background execution:

Client
   |
   v
Submit Run
   |
   v
Run ID
   |
   v
Background Execution
   |
   v
Execution Result

Use streaming when the client wants to consume output during an active request. Use background runs when execution should be submitted and queried independently. The v0.1 background execution model is process-local. It is not a distributed durable queue.

Streaming and Authentication Errors

Authentication is checked before the protected streaming endpoint can be used. A missing or invalid API key results in an authentication error rather than a stream. For example:

Client
   |
   | Missing/invalid key
   v
BindAI API
   |
   v
401 Unauthorized

A valid request proceeds to agent execution.

Streaming and Agent Errors

A validly authenticated request can still fail during execution. Potential failure sources include:

Agent configuration
Model providers
Tools
Connections
External services
Application code

Streaming should therefore be treated as an execution response, not as a guarantee of successful completion. Clients should handle connection failures and incomplete streams appropriately.

Streaming and Tools

An agent may use tools during execution. Conceptually:

Client
   |
   v
Streaming Agent
   |
   +---- Model
   |
   +---- Tool
          |
          v
      External Service

The streaming endpoint provides the HTTP output stream for the agent execution. The current v0.1 streaming API should not be treated as a structured event stream for every internal model, tool, workflow, or connection event. For internal execution observability, BindAI provides the runtime event system and EventRecorder.

Streaming and Observability

BindAI’s runtime event system can record structured execution events independently of the HTTP streaming response. Relevant event categories include:

Agent started
Agent finished
Model requested
Model responded
Tool executed
Workflow events
Node events
Memory events
MCP connection events

This creates two distinct concerns:

HTTP Streaming
      |
      v
Client-facing agent output

and:

Runtime Events
      |
      v
Application observability

The streaming response should not be assumed to contain all runtime events.

Content Type

The current v0.1 streaming route returns a text-based StreamingResponse. Clients should consume it as streamed text. The current implementation is not documented as a Server-Sent Events (text/event-stream) protocol. Applications should therefore avoid assuming SSE-specific framing or event fields. If a future BindAI release introduces a structured streaming protocol, that protocol will be documented separately.

Streaming Protocol Scope

The current v0.1 streaming implementation intentionally provides a simple HTTP streaming interface. It does not define a complete structured event protocol for:

Token metadata
Tool-call events
Tool results
Structured model events
Usage events
Reasoning events
Workflow events
Connection events

Applications requiring these details should use the BindAI runtime event system or implement an application-specific event layer. A future release may provide a richer structured streaming protocol.

API Documentation

FastAPI automatically exposes API documentation for the BindAI API application. When running locally, the documentation is available through the standard FastAPI documentation routes. For example:

http://localhost:8000/docs

The OpenAPI schema is also available from the running API. The generated documentation reflects the installed BindAI API version.

Testing Streaming

The API package includes tests for the implemented agent API, including streaming behavior. Run the API test suite with:

uv run pytest packages/bindai-api/tests -q

Streaming behavior should be validated for:

Endpoint availability
Authentication
Agent lookup
Request validation
Streaming response behavior
Returned agent output

Provider-specific streaming behavior should be tested separately where required.

Local Development

Start the API locally with:

uv run uvicorn bindai_api.app:app --host 0.0.0.0 --port 8000

Then call:

POST http://localhost:8000/api/v1/agents/{agent_name}/stream

with:

Authorization: Bearer <API key>
Content-Type: application/json

and:

{
  "message": "Hello from BindAI."
}

Production Considerations

When deploying streaming endpoints publicly, consider:

HTTPS
Reverse-proxy buffering
Request timeouts
Connection timeouts
Client disconnects
API authentication
Rate limiting
Provider latency
Concurrent streams
Resource usage
Monitoring

Streaming requests can remain open longer than ordinary request/response operations. Infrastructure limits should therefore be configured for the expected workload.

Current v0.1 Scope

BindAI v0.1 provides:

Agent streaming through the REST API
Bearer API-key authentication
HTTP streaming responses
Python HTTP client compatibility
JavaScript Fetch compatibility
cURL compatibility
FastAPI/OpenAPI documentation

The current implementation is intentionally lightweight. It does not provide a standardized structured streaming event protocol. It does not turn the streaming endpoint into a durable background execution system. It does not provide distributed streaming coordination.

Future Streaming Capabilities

Future BindAI releases may expand streaming with capabilities such as:

Structured streaming events
Token metadata
Tool-call events
Tool-result events
Usage information
Execution identifiers
Stream cancellation
Richer error events
Server-Sent Events support
WebSocket support
Structured workflow streaming

These capabilities are not required for the BindAI v0.1 release.

Summary

BindAI v0.1 exposes agent streaming through:

POST /api/v1/agents/{agent_name}/stream

Requests require:

Authorization: Bearer <API key>

and:

{
  "message": "Your message"
}

The endpoint returns a text-based streaming HTTP response that allows clients to consume agent output incrementally. Streaming is useful for interactive applications, but it should not be confused with background execution or runtime observability. The current architecture separates:

Streaming
    |
    v
Client-facing agent output

Background Runs
    |
    v
Long-running application execution

Runtime Events
    |
    v
Observability and execution inspection

This lightweight streaming API provides the foundation for interactive BindAI applications while leaving richer structured streaming protocols for future releases.

​Streaming

​Streaming Architecture

​Endpoint

​Authentication

​Request Body

​Streaming Response

​Streaming with cURL

​Streaming with Python

​Streaming with JavaScript

​Streaming vs Normal Execution

​When to Use Streaming

​Streaming and Model Providers

​Chunk Handling

​Buffering

​Reverse Proxies

​Connection Lifetime

​Client Disconnects

​Streaming vs Background Runs

​Streaming and Authentication Errors

​Streaming and Agent Errors

​Streaming and Tools

​Streaming and Observability

​Content Type

​Streaming Protocol Scope

​API Documentation

​Testing Streaming

​Local Development

​Production Considerations

​Current v0.1 Scope

​Future Streaming Capabilities

​Summary

Streaming

Streaming Architecture

Endpoint

Authentication

Request Body

Streaming Response

Streaming with cURL

Streaming with Python

Streaming with JavaScript

Streaming vs Normal Execution

When to Use Streaming

Streaming and Model Providers

Chunk Handling

Buffering

Reverse Proxies

Connection Lifetime

Client Disconnects

Streaming vs Background Runs

Streaming and Authentication Errors

Streaming and Agent Errors

Streaming and Tools

Streaming and Observability

Content Type

Streaming Protocol Scope

API Documentation

Testing Streaming

Local Development

Production Considerations

Current v0.1 Scope

Future Streaming Capabilities

Summary